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About the research 


This report is the result of a research project initiative of NCVER and the UNESCO-UNEVOC 
International Centre for Technical and Vocational Education and Training (TVET) in Bonn (Germany). 
The initiative aimed to facilitate research collaboration in the UNEVOC Network towards development 
of tools that can help improve TVET funding mechanisms through evidence-based research and 
strengthen capacities in this area. 

This report presents a conceptual framework for understanding the return on investment (ROI) 
equation in TVET from different stakeholder perspectives. 

The framework uses three main stakeholder groupings - individuals, business and the economy. 
Although the framework separates these three perspectives, they are not independent of each other. 
There are flow-on effects. To provide a complete ROI picture both economic and social impact 
dimensions are featured. Understanding the interaction between the economic and social benefits is 
important in assessing the true and full value of TVET. The key indicators were selected on the basis 
of their usefulness, practicality and capacity to value-add along with the ability to apply to different 
types of training and contexts. 

Key messages 

The authors highlight the following key observations: 

■ The key types of ROI for individuals arising from TVET are primarily employment and productivity 
supporting higher wages. Attainment of employability skills and improved labour force status are 
also highly valued job-related returns. Non job-related indicators focus on well-being such as self¬ 
esteem and confidence, foundation skill gains, along with social inclusion and improved socio¬ 
economic status. 

■ The key indicators of ROI for employers arising from TVET cover employee productivity, business 
profitability, improving quality of products and services and business innovation. Businesses 
operate similar to small communities and as such generate social and environmental benefits. In 
particular employee well-being, employee engagement (which reduces absenteeism and staff 
turnover), a safe workplace and environmental sustainability practices are key non-market 
indicators of business returns. 

■ The key indicator of ROI in the economy from TVET is economic growth. This relates to labour 
market participation, reduced unemployment rates and a more skilled workforce. TVET returns to 
education and training, bring other benefits to society, including improved health, social cohesion 
(increased democratisation and human rights), and improved social equity particularly for 
disadvantaged groups and strengthens social capital. 

The report recognises that analyses of ROI in TVET can result in highly variable estimates; and that it 
is particularly difficult to untangle the financial and non-financial benefits of training. Further, the 
ready availability of data to populate such a framework is a challenge for it to gain greater practical 
value and allow estimates of ROI across economies. 


Craig Fowler 

Managing Director, NCVER 
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o Introduction 

Technical and Vocational Education and Training (TVET) is seen as an important strategy in 
contributing to equitable, inclusive and sustainable economies and societies. The United Nations 
(2015) lists one of its sustainable development goals as to ‘ensure inclusive and equitable quality 
education and promote lifelong learning opportunities for all’. However, this comes with challenges 
for the funding and financing of TVET systems internationally and also for providing evidence for the 
return on investment (ROI) in TVET. 

Providing information on ROI in TVET is important as it provides governments and funders of the 
system with analytical information on the performance of the system and further provides 
justification for the expenditure on TVET. Information on ROI is also useful at the level of the 
enterprise and the individual. However, the measurement of ROI is not straightforward and thinking 
through what is involved in the ROI calculation can give a better understanding as to what type of 
information and data is required to calculate the measure. This may also vary depending on the 
context of the country’s TVET system. 

Hence, this report presents a conceptual framework for measuring ROI in TVET that can be tested in 
international contexts. It builds on previous work done as part of a larger collaborative project by 
UNESCO-UNEVOC in association with the National Centre for Vocational Education Research (NCVER) in 
Australia, and other UNEVOC Centres in the Asia-Pacific region. 

The aim of the collaborative project is to investigate measurement of ROI across different contexts 
including across varying countries. The longer-term aim of the ROI project is to equip organisations in 
various countries to be able to systematically investigate evidence of ROI in TVET and to engage a 
range of stakeholders in this process. Part of this is the development and testing of a suitable ROI 
framework that can be applied internationally. There may well be variations between countries in 
terms of priorities regarding the costs and benefits of TVET. There will almost certainly be variations 
in terms of the data that is available to measure ROI in TVET. 

This report firstly summarises some of the main issues that need to be thought through in measuring 
ROI. It then introduces an analytical framework that looks at the ROI equation from a range of 
perspectives, including economic and social and for different stakeholders; including individuals, 
businesses, governments and societies. 

Definitions 

For the purposes of this report: 

Return on investment or ROI refers to a measure of the benefit of an investment relative to the cost 
of that investment. So in the TVET context, ROI is the benefits derived by individuals, firms and 
nations from investing in training (VET Glossary 2016). 

Returns to education refer to the individual gain from investing in more education, especially 
focussed on the relationship between education attainment and earnings. However, for consistency 
and simplicity, this report tends to use the terminology of Return on Investment or ROI. 

Technical and Vocational Education and Training or TVET comprises education, training and skills 
development for a wide range of occupations. It can take place in secondary school and tertiary 
education and includes work-based learning and continuing education and training. 
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Types of returns 

A comprehensive study by Griffin (2016) explored the existing international literature around the ROI 
from different perspectives and stakeholder groups. 

Broadly, at the economic level, research on the ROI in TVET falls into two broad categories: 

■ determining the ROI for spending that has occurred 

■ investigating the potential return should spending/funding be altered. 

Both of these approaches have demonstrated the value of TVET to the economy through increases in 
employability and, to a lesser degree, increases in productivity. 

A summary of the types of returns include: 

■ For the individual, higher-level VET qualifications are consistently demonstrated to provide a good 
return on investment. The individual returns from TVET are mostly generated through increased 
participation in the workforce. Lower-level qualifications consistently resulted in lower financial 
returns, although these qualifications may result in other benefits, such as further study or 
improved self-esteem and wellbeing. 

■ For individual businesses, analyses of ROI in training result in highly variable estimates. This may 
be because the methods used appear to be more suited to industries where increases in 
productivity are easier to define and measure (such as in manufacturing, where some very high 
returns were reported, compared with service-based industries). It is particularly difficult to 
untangle the financial and non-financial benefits of training to business, as many improvements, 
such as reduced staff turnover, absenteeism, and positive changes to workplace culture, may also 
result in economic pay-offs for the business. 

■ For societies, in addition to productivity gains, education and training has also been shown to bring 
other, non-financial benefits to society such as improved health and reduced national crime and 
drug use, greater social cohesion and the potential for poverty reduction. 

In support of this research, the benefits of training can also be grouped and illustrated in various 
ways. 

The European Centre for the Development of Vocational Training (CEDEFOP) (2011) considers the 
benefits of TVET across two dimensions; economic and social. These two dimensions are then further 
grouped by three levels; micro, meso (or intermediate) and macro with the micro level approximately 
according with the level of the individual and the macro level with the level of the country (figure 1). 
Thinking about these benefits and which ones are most important in the context of a particular 
country’s TVET system assists in considering how these benefits can be most effectively measured. 
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Figure 1 The benefits of vocational education and training. 
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Source: CEDEFOP 2011, Research paper 10: The benefits of vocational education and training, Publications Office of the 
European Union, Luxembourg. 
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@ Background on measuring ROI in TVET 

Previous work (Griffin 2016; Schueler 2016) indicates that measuring ROI in TVET is a very complex 
matter. Firstly, there are different dimensions of outcomes of TVET to consider. For instance, 

Marope, Chakroun and Holmes (2015) in summarising arguments for investment in TVET across 
countries consider three arguments for the investment in TVET. These are economic growth, social 
equity and sustainability. Within different countries one or more of these arguments may be 
emphasised to a greater or lesser extent and this will naturally have an influence on which measures 
of ROI they should focus on. 

Following on from this context is very important. The political, economic and education system of a 
country, and the stakeholders involved all have influence on which aspects of ROI in TVET are 
important. This means that what is measured in one country in terms of ROI might not necessarily be 
exactly the same as in another country, although there may well be some common baseline as to what 
is measured. 

Another complexity is the availability of data to measure TVET. Data that exists may not necessarily 
be in a form that is readily useful for the measurement of ROI, or more pointedly, may not exist at 
all. Participants in a virtual conference on ROI run by UNESCO (UNESCO-UNEVOC 2016) noted several 
aspects to the challenge of having data for ROI. These include having ROI in mind when setting up 
data collection for TVET, coordinating stakeholders that may provide data, setting up appropriate 
Management Information Systems, and broader approaches to the collection of data (for example, 
qualitative data if need be). 

Given the above considerations and before arriving at an evaluative framework for ROI in TVET, a 
number of issues need to be thought through. This background explores these issues and is based 
largely on Schueler (2016). 


Issues in developing the ROI evaluation framework 

The key elements constituting an evaluation framework based on a review of the literature are shown 
in figure 2. The various quadrants in the framework are discussed below. 

Figure 2 ROI Evaluation Framework elements 


Context Guiding Principles 

Scope ROI Model 

Purpose Indicators 


ROI 


Costs 
Benefits 
Impact factors 


Data collection 
Data validation 
Data conversion 
ROI calculation 
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Context, scope and purpose 

The context, scope and purpose form the foundation of the framework. Developing a return on 
investment (ROI) evaluative framework requires understanding the TVET context. Furthermore, an 
aggregated cost-benefit analysis is challenged by variations in TVET systems and ROI methodology 
(OECD 2008). It is context specific and impacts on the definition and calculation of TVET costs and 
benefits. Hence the outcome from any ROI analysis tends to be relative and restricted to a specific 
environment. 

In terms of scope, there are multiple levels of stakeholders including individuals, enterprises and 
economies. Within each of these different levels of stakeholders are multiple dimensions of ROI. 
These include the economic, social and environmental measures of ROI. Defining a specific statement 
of scope keeps this measurement practical and focussed. (For example, the social returns to 
organisations from workplace literacy training is specific and clear). 

Clarity of purpose is integral to implementing a Return on Investment Framework. This maintains 
focus and helps to specify the parameters. Studies have used ROI for various reasons. These include 
business improvement through supporting new technologies and improving workforce efficiency, 
workplace health and safety (Brown et al. 2015) and as part of funding agreements (IPP 2012). 

Guiding principles, models and indicators 
Developing guiding principles 

The guiding principles ensure that a consistent and standard frame of judgement is applied to the ROI 
evaluation. The guiding principles consider the following: 

■ The ROI model or method to be adopted. This must be customised, fit for purpose and add value. 
It requires an overarching clarity of purpose. The model should measure factors that are specific 
and relevant to the context. 

■ The implementation of the ROI model. There are a few issues to consider here such as whether it 
is practical and will provide information that meets the needs of stakeholders. The model also 
needs to cater for a range of measures and data sources, a variety of training types, and whether 
it can be applied before, during and after training. 

* The development of the methodology and data collection instruments processes and instructions. 
This includes ensuring that the data collection instruments are capable of being customised to 
particular context while being specific enough about the data that is required. In addition, they 
should place minimal load on the stakeholders that need to administer them. 

■ The compilation of credible evidence about the impact of training. Firstly, the data has to be of 
sufficient quality. The ensuing analysis then should be scientifically valid and address the fact 
that training may not be the only factor that explains changes in performance or outcomes. 

Approaches to measuring ROI 

The rest of the information, divided into quadrants in the evaluative framework, can be usefully 
represented by a flowchart or ‘decision tree’. Figure 3 presents this process of measuring ROI within 
the evaluative context. The various components of this flowchart are discussed in the sections that 
follow. 
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Figure 3 Measuring ROI process 
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Models of ROI 


TVET research studies use various models to determine ROI. These models include measuring 
economic and social impact (SROI). Some examples of ROI models are shown in table 1. Different 
models apply to different situations and may suit specific types of data. The decision to include 
economic and social returns will influence the selection of the ROI model along with the choice of an 
evaluative or forecasting perspective. The best fit model enables customisation, adds value and 
measures factors that matter and are specific. 


Table 1 Return on Investment models 


ROI model 


Description 


Cost-Benefit Analysis (CBA) 

Internal Rate of Return (IRR) 

Kirkpatrick/Phillips Evaluation 
Model 

Net Present Value (NPV) 
Return on Expectation (ROE) 

Social Return on Investment 
(SROI) 


Assigns monetary value to costs of the training program to determine the cost-benefit 
ratio. 

Rate of interest that equals the returns from an investment to the cost of the 
investment. 

4 Levels of Evaluation - Reaction, Learning, Behaviour, Results plus Level 5 ROI that 
converts 4 th level to monetary value. 

Compares the value of money now with the value in the future. 

Estimates returns to training relative to stakeholder expectations. Uses surveys and 
interviews. 

Stakeholder driven evaluation with cost-benefit analysis and strong focus on social 
impact. 


Note: Derived from several sources and studies. 


Some examples of specific studies of ROI that have been conducted are shown in table 2. Most of 
these examples appear in reviews of the links between education and training and its benefits by 
Griffin (2016) and the Australian Workforce and Productivity Agency (2013). There are studies in the 
table that look at returns from the perspective of the economy, firms and individuals. The table 
provides indications of the data that was used, the econometric or otherwise quantitative models 
applied and the main results from the analysis for each of the examples. 


Table 2 Examples of studies investigating the return on investment of training to the economy, firms and 
the individual 


Study 

Methodology used 

Findings 

Two studies investigating the return on investment of training to the economy 

Canton 2007 (cross-country) 

Analysis used a macro¬ 
version of the Mincer 
relationship between 
education and wages at the 
individual level. Data source 
was estimates of school 
attainment by Cohen & Soto 
(2001) for 95 countries split 
by 5 year age groups in 10 
year intervals. 

An increase of one year of average education level of the 
labour force increases labour productivity by 7-10% in the 
shortterm and 11-15% in the longer term. There was also 
some evidence of spillovers in the sense of the human 
capital stock increasing prospective economic growth. 

Independent Economics 2013 
(Australia) 

Cost-benefit analysis to 
estimate the return of 
increased funding in VET. 

The committed 5.6% increase in funding was predicted to 
result in an 18% internal rate of return to the economy. 

Three studies that aimed to investigate methods to assess the relationship between training and productivity 

Blandy et al. 2000 (Australia) 

Surveys (based on larger 
international examples) and a 
small number of in-depth case 
studies. 

10% increase in training resulted in a 1% increase in 
productivity growth. 

Training quantity and quality were positively associated with 
profitability. 

Dearden, Reed and Van 
Reenen 2005 (UK) 

Analysis of constructed panel 
study of firms using the 
econometric estimation 
technique General Method of 
Moments (GMM). 

They found that a 1% increase in training in firms resulted in 
about a 0.6% increase in productivity; double the 0.3% 
increase in wages. 
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Bernier and Cousineau 2010 Analysis of the longitudinal 
(Canada) Statistics Canada Workplace 

and Employee Survey (WES) 
using a Cobb-Douglas 
function within a distributed 
lag estimation framework. 


There are positive effects of training on productivity spread 
out over a three year period. In addition, they found 
interactions between investments in capital and investments 
in training. A 10% increase in investments accompanied by 
expenditure in structured training per employee resulted in 
an average increase of 0.6% in corporate productivity the 
following year compared to a company that did not invest in 
capital complementary to training. 


Four studies that aimed to investigate methods to assess the relationship between training and the individual level 


Green and McIntosh 2006 
(UK) 


CEDEFOP 201 Id (cross¬ 
country) 


Chesters, Ryan and Sinning 
2013 (Australia) 


Polidano and Ryan 2016 
(Australia) 


Uses wage equations at the 
individual level and 
productivity equations at the 
industry level. Data used was 
from the Labour Force Survey 
in the UK from 1996-2005 
(including the National Adult 
Learning Survey as a 
supplement to the LFS in 
2003) and comparative data 
from Eurostat for other 
European countries to 
examine non-certified 
learning. 

The study used a variety of 
comparable data sets from 
across the EU. Multi¬ 
regression analysis 
techniques were used to 
examine the effect of 
vocational education and 
training on wages and 
employment status. 

The research used the Survey 
of Aspects of Literacy (SAL) 
and the Adult Literacy and 
Life Skills (ALLS) survey to 
investigate the income returns 
of literacy skills in the 
Australian Labour market. 
Analysis uses a modified 
standard human capital 
earnings function that adds 
literacy skills as a determinant 
of earnings. 

The research used the HILDA 
longitudinal dataset to 
examine the long-run effects 
of completing VET. Fixed 
effects regression methods 
were used to estimate long- 
run effects of obtaining new 
qualifications, and also 
obtaining further qualifications 
at the same, higher or lower 
level than the previous 
qualification. 


Individuals who undertake non-certified learning earn about 
5-6% more in wages than those who do not. 

No significant relationship was found between rate of 
non-certified learning and productivity for industries. 

It is the undertaking of the non-certified learning that is 
important; not the period of learning. 


The returns for an extra year of tertiary and equal to the 
return of one extra year of initial VET. Education on wages 
and employment are about 7% for both males and females. 
Adjusted returns to account for the short duration of the 
training (17 weeks on average) gives yearly returns for VET 
of 10% for men and 7% for women. 


There is a positive association with income for observed 
literacy skills and educational qualifications for full-time male 
and female employees. 

Income was found to increase with literacy skill level within 
defined broad education levels. 

There are higher returns to literacy skills for highly educated 
workers as opposed to workers with lower levels of 
education. However, for those workers with lower to medium 
education, the returns to literacy skills have increased over 
time for some cohorts. 


The effects (for example labour market outcomes) of 
obtaining a VET qualification are often larger for females 
than for males. The longitudinal data showed stability of 
effects over time with significant effects found in the first 
year after course completion remaining up to five years 
later. Further completed qualifications not higher than the 
previous qualification did not consistently result in better 
labour market outcomes in this study. 


Dimensions of ROI 

There are many layers and dimensions to ROI measurements. They are different for each stakeholder. 
There are economic and social aspects. Economic impact is more easily measured but it is the social 
impact that completes the whole ROI picture. The studies indicate that the social implications of 
training are most important to understand as they provide a true value of training that is often 
neglected in TVET research (due to difficulty in measuring). Table 3 shows a sample of ROI indicators 
for individuals, employers and the economy. 
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Table 3 ROI indicators by stakeholder 


Individuals 

Employers 

Wider community 

Job related 

Market 

Economic 

Employability 

Productivity 

Labour market participation 

Productivity - skill gains 

Efficiency 

Labour force productivity 

Earning capacity 

Employee workplace literacy 

Increasing the tax base 

Foundational skills - literacy 

Employee skill gains 

Gross domestic product (GDP) 

Training pathways - vocational/higher 
education. 

Business innovation 


Non-job-related 

Non market 

Social 

Wellbeing 

Organisational culture 

Social cohesion 

Engagement 

Motivated workforce 

Social inclusion 

Satisfaction 

Employee well-being 

Health and wellbeing 

Self-esteem/confidence 

Employee work practices 

Crime reduction 


Note: Derived from several sources and studies. 


For individuals, many studies report on economic impact. The two main training influences are 
through improved employability and increased productivity (Independent Economics 2013). However, 
there are gains unrelated to jobs that are also a result of training, reflected in self-confidence, 
well-being and engagement (NVEAC 2011). The ROI measurement outcome is also influenced by the 
intent of the individual with reasons for undertaking training ranging from promotion, 
vocational/higher education pathway to personal development. 

Organisations and employer’s training outcomes are commonly analysed by productivity gains and 
efficiency (AWPA 2013). In addition, there are non-productivity returns through employee well-being, 
work practices and organisational culture. The reasons for committing to TVET training also goes 
beyond productivity to legislative and licensing requirements, introducing new technologies (Smith et 
al. 2009) and other business improvements. 

Data collection 

There are two main steps within the actual data collection process. The first is preparing information 
to guide the process. This includes, for example, developing data collection instruments and 
supporting documentation, defining data elements to be collected, and identifying possible data 
sources in supporting instructions and documentation. 

The second step is establishing existing data sources. This includes identifying types of quantitative 
and qualitative data sets such as national data collections, administrative datasets, longitudinal 
studies, surveys, interviews and case studies. Of further importance is determining and documenting 
the completeness of the data, its quality, gaps in the data, where the data will be collected, and the 
conceptual underpinnings of the data. 

Once the data has been collected it may need to be converted into monetary terms. Not all the data 
that is collected will necessarily be in monetary form; particularly the less tangible costs and benefits. 
Therefore consideration needs to be given as to how the data can be converted to monetary or 
quantifiable values (if possible/required) and what is an appropriate conversion method and process. 

The data will also require validation before statistical analysis for ROI is conducted. That is, the data 
being used in the ROI calculation should measure what it is supposed to be measuring. Furthermore, 
the data needs to be reliable in terms of being reproducible, consistent and accurate. Then an 
analysis for return on investment can take place using an appropriate statistical approach. For 
example, regression or multivariate analysis has been applied to ROI data to control for variables to 
ensure the ROI outcome is a direct result of training. 
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Costs and benefits 


The costs and corresponding benefits of TVET are critical to ROI. Related to this is a consideration of 
how these can be impacted by other factors, for example, the state of the labour market. 

Training costs 

Training costs and total investment are generally underestimated. There are two categories of costs — 
direct and indirect costs. These costs differ by stakeholder type and attributes of the specific training 
program. TVET costs are paid by students, businesses, industry, training providers and the community, 
for which data may be difficult to collect. Table 4 provides an example of individual and employer 
costs associated with training. 


Table 4 Training costs by stakeholder 


Individual 

Business/employers 

Government 

Direct costs 

Direct costs 

In addition to public expenditure 

Tuition 

Course costs for employee 


Books and materials 

Salary of staff while on training 


Equipment (for example, computer) 
Childcare 

Travel/parking 

Special fees (for example, library) 

Course design and development 


Opportunity costs 

Intangible costs 

Indirect costs 

Foregone or reduced earnings while 

Loss of productivity while trainees are 

Payroll tax rebates 

studying 

attending course 

Workforce development programs 


Induction costs 

Completion bonuses of employers of 

Non-completion costs 

Costs of replacing employee while 
attending course 

Higher wastage rates until the trainee is 
fully proficient 

Missed opportunity costs 

apprentices 


Note: Derived from several sources and studies. 


Direct costs are more easily measurable. For individuals these expenses can vary. Financial costs may 
differ between courses, providers and with concession (Watson 2011) where subsidies and student 
vouchers provide external financial support. Requirements of the industry and field of study may 
require additional equipment, materials and for some protective clothing to be purchased. As an 
employee, these tuition costs are paid by the employer. However accessing cost data at the business 
level may be difficult (AWPA 2013). 

Indirect and intangible costs are not as clearly quantifiable or easily captured. For example older 
students may need to pay for childcare or forgo employment for a period of time and absorb loss of 
income. Employers also bear the costs of not having adequately skilled employees that are not fully 
proficient at their job, lost time while employees are in training and increased workloads in their 
absence (NCVER 2013). Intangible costs may also be difficult to convert into monetary terms. 

The point in time when training costs data are collected impacts on the resulting return on 
investment. Costs can be measured over different periods of time — before the training, upfront, 
during the training or part of on-going costs (OECD 2008). The point in time capture of cost data is an 
important factor of the evaluative or forecasting ROI analysis. 
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Training benefits 


There are two categories of benefits — market and non-market. In the workplace, for example, these 
refer to job related and non-job-related outcomes. Table 5 illustrates some of these benefits for 
individuals, organisations and the economy. 

Market benefits are directly measurable and relative to the stakeholder group. The main benefit of 
TVET that influences pre-tax earnings of individuals is improved employability (Long & Shah 2008), 
and for the economy, increased participation in the workforce. Training ‘pay-off’ to individuals also 
varies depending upon whether the training can be considered preparatory (such as initial VET) or 
higher-level (Karmel & Fieger 2012). 

Non-market benefits are not so easily quantifiable. Employee social and well-being aspects or business 
workplace literacy, safety and workforce flexibility are more difficult to measure. Outcome measures 
tend to have an economic focus, excluding community and personal outcomes that are more difficult 
to quantify (OECD 2008). A model that takes both market and non-market benefits is recommended 
(CEDEFOP 2013). 


Table 5 Benefits of TVET training by stakeholder 


Individuals 

Employer 

Economy 

Job related 

Market 

Tangible benefits 

Higher employability 

Productivity 

Higher employability 

Employment 

Sales & profitability 

Increased participation in the workforce 

Higher salaries 

Customer service and satisfaction. 

Decrease in unemployment levels 

Higher savings levels 

Occupational health & safety 

Productivity gains 

Improved working conditions 
Professional mobility 

Productivity (highly skilled) 

Quality product & services 

Saving on material & capital costs. 

Higher skilled workforce 

Non-job-related 

Non-market 

Intangible benefits 

Higher education pathway 

Motivated workforce 

Improved health 

Pathway to further study 

Improved organisational climate and 

Improved environment 

Improved self esteem 

culture. 

Reduced national crime 

Communication skills 

Increased literacy in workplace 

Increased social cohesion 

Engagement 

Employee skill gains 

Increased social inclusion 

Improved problem solving 

Employee well-being 

Strengthened social capital 

Improved health & wellbeing 

Improved economic standards of living 
Life satisfaction 

Social inclusion 

Employee workplace practices 

Active citizenship 

Technological change adaptation 


Note: Derived from several sources and studies. 


Benefits also vary depending on the stakeholder’s perspective. Table 6 illustrates tangible and 
intangible benefits of training to an employee and those of the employer. The table indicates the 
individual (employee) benefits cover improved earnings, skills and work practices while the employer 
benefits are concerned with productivity, compliance, safety and quality. 
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Table 6 

Benefits of Training for Employees and Employer 


Benefits 

Employee 

Employer 

Tangible 

Improved employee pay 

Improved language & literacy 

Improved technical skills 

Increased use of new technologies 

Improved workplace practices and procedures. 

Increased productivity and efficiency 

Increased sales and profitability 

Improved product quality & services 

Improved customer service and satisfaction levels. 
Improved Occupational Health and Safety 

Intangible 

Social and well-being: 

Improved self-confidence & morale 

Reduced stress 

Improved motivation 

Improved work ethic 

Improved physical and mental health 

Job satisfaction 

Better management and employee workplace 
relations 

More co-operation among employees 

Reduced internal conflicts 

Developing a learning culture 

Supporting social cohesion and inclusion 


Source: Adapted from Barker 2001 and Moy and McDonald 2001 in Brown et al. 2015, Workplace Literacy Pays, ACER. 


The benefits may arise at different points in time and may extend well beyond the completion of the 
training. The estimation of ROI can relate to a time period during a training program, at its 
completion or long after the event (OECD 2008). 

Table 7 shows a comparison of the short and long term benefits of training to the individual, 
organisations and the economy. Medium to long term benefits such as mobility or the capacity to 
upgrade skills later in life are more difficult to quantify (OECD 2008). 


Table 7 Short-term and long-term benefits of training by stakeholder 


Benefits 

Individual 

Employer 

Economy 

Short-term benefits 

Employment opportunities. 
Increased earnings levels. 
Work satisfaction. 

Higher productivity from 
trained workforce. 

Saved costs from recruiting 
external skilled workers. 
Improved quality of products 
and services. 

Improved customer 
satisfaction levels. 

Reduced reliance on 
welfare. 

Social cohesion. 

Long-term benefits 

Greater employee flexibility 
and mobility. 

Lifelong learning. 

Reduced employee turnover. 
Improved safety record. 

Better workplace relations. 

Productivity gain from 
educated workforce. 

Increase in tax income from 
higher earnings. 


Source: Costs and Benefits in Vocational Education and Training 2008, OECD adapted from Barker 2001 and Moy et al 2001. 


Other factors impacting on ROI 

There are a range of factors that can influence ROI apart from costs and benefits. Table 8 illustrates a 
summary of determinants that range from individual characteristics of age and level of qualification 
to the size of an organisation to the quality of the trainer. 

If, for example, training works better in the workplace than the classroom; in collaboration rather 
than self-directed; associated with a specific application (for example, new technologies) or for those 
already possessing sound foundation education and skills - then we need to isolate and control for 
these variables to ensure that the ROI outcome is a direct result of the training. 

In particular, there are specific factors that impact on ROI for each stakeholder group. These are 
educational background and qualification level of the training program for individuals; industry type, 
organisation size and profit/not-for profit sector type for enterprises and the profile characteristics of 
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the population for the economy. Of particular note for all stakeholder groups is the type of training 
program and the field of education or industry and the impact on ROI. 


Table 8 Types of factors that impact on the ROI results 


Category 

Description of factor 

Stakeholder characteristics 

Individual’s demographics - age, educational attainment, level of schooling (often 
different calculations), apprentice. 

Employers/business - size of organisation, industry, private/public, (type of 
employee training) 

Training status 

Qualification completers or module completers 

Part-time or full-time status 

Reskilling or upskilling 

Training Programs/Course 

Qualification level 

Non accredited versus accredited training 

Training level - for example, foundational - literacy & learning 

Industry/field of study 

Types of training - leadership, management, innovation, apprenticeship 

Initial TVET or on-going training, (for example, apprentices/trainees) 

Highly specific or general training (more transferable) 

Training Context 

TVET in schools (teacher quality, student engagement, employer relationships 
(relevant/effective) - training pathways, material resources. 

Workplace training versus classroom training 

Training in partnership versus self-directed. 

Training Provider 

Private vs public 

Quality of trainer 

Quality of resource materials 

Labour market 

Demands for skills, labour market regulations, trade union influences 


Source: Adapted from Measures of Success Research Framework 2011. Human Resources and Skills Development Canada (In 
Brown et al. 2015, Workplace Literacy Pays, ACER). 


Issues with data collection 

Any ROI model must be supported by useful and practical indicators. However, there are various 
considerations regarding data for ROI measures. This is in addition to initially establishing the 
availability of data as discussed earlier. 

One consideration is the integrity and credibility of the data. This refers to the level of data accuracy, 
validity (is it measuring what it is supposed to measure) and reliability (is it consistent and 
reproducible). There is also a need to recognise that the data validity and reliability may vary with 
the level of data aggregation. 

Another consideration is data comparability. In order to compare data there needs to be consistent 
definitions across data sets and the need to establish the basis for data comparisons. This includes 
establishing which data is directly comparable and which is not. 

The data also needs to be transparent. This means that its conceptual underpinnings are carefully 
explained. In addition, a description of data quality and completeness is necessary along with 
highlighting any gaps or anomalies. 

In summary, the measurement of ROI requires context, scope and purpose with guiding principles that 
form a standard frame of judgement and a practical ROI model that is customised, fit for purpose and 
measures what matters. It identifies relevant costs and benefits of TVET and how these can be 
impacted by other factors. It supports useful indicators, practical measures and quality data and then 
applies scientifically valid techniques which seek to address the influence of other variables. 
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© The ROI analytical framework 

This section outlines the main elements of an ROI evaluative framework. 

Background 

A review of the literature on ROI consistently indicates that ROI is context specific to the stakeholder 
and relative to the environment (OECD 2008). Furthermore, the differences between the area of 
focus, demographics and methodology add to the variable outcomes. Although the measurement of 
ROI is both diverse and complex, identifying key indicators to formulate a conceptual framework 
requires an overarching structure that supports a practical approach with broad application. 

The ROI framework presented in this section is based on the extensive analysis of existing research 
approaches in this area. A discussion of the main features of the model aim to provide a 
comprehensive guide to the perspective, dimensions and objectives that underlie the framework 
along with the high-level indicators that direct attention to capturing measurable outcomes. 

Stakeholder perspective 

This ROI framework has three stakeholder perspectives — individual, business and the economy. In the 
literature these are the three main stakeholder groups that focus on ROI from TVET. 

■ There are many studies that focus on returns to the individual with a particular focus on the 
relationship of qualification levels with employment and wages. This area has been well researched 
and supported by extensive data (Karmel £t Nguyen 2006; Leigh 2008; Noonan et al. 2010). 

■ The business tier covers employees, employers, individual businesses and industry. In this sector, 
training is highly specific for target groups and it is often through case studies that business 
outcomes are derived, although Bartel (2000) notes that large sample surveys of firms and 
econometric type case studies of one or two companies are also used. The return on investment 
maps across operations, profitability, product/services and human resources. 

■ At the macro level, the impact of TVET on the economy has been studied through economic 
modelling and other predictive approaches to measure economic growth (Independent Economics 
2013). Analysis of varying population profiles also provides insight into social returns. 

Although the framework separates these three perspectives, they are not independent of each other. 
The dynamic relationship between stakeholder groups can have flow-on effects. 

Economic and social dimensions 

To provide a complete return on investment picture, both economic and social impact dimensions are 
featured in the framework. 

The majority of studies focus on economic returns to stakeholders. These include financial returns to 
individuals in the labour market (Karmel Et Fieger 2012; Long & Shah 2008), productivity and 
profitability to business (Cedefop 2013) and economic growth indicators (Independent Economics 
2013). However the literature (Stanwick et al. 2006; Cedefop 2011a; IIP 2012) also points to a great 
interest in the social outcomes of TVET which are not always so easy to measure as economic 
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markers. More recently, the health and well-being of individuals, employees and society is capturing 
the attention of ROI research. 

For the purposes of this framework, economic indicators relate to job-related outcomes for 
individuals, and market indicators for business and economic growth for the economy. Social 
indicators refer to non-job-related outcomes for individuals, and non-market indicators for business, 
for example, employee engagement and social returns of health and well-being in society. 

Most importantly, understanding the interaction between the economic and social benefits is vital in 
assessing the true and full value of TVET. The dynamics of social returns to directly impact economic 
drivers is of notable interest. 

Environmental impact is also of primary importance to this discussion across stakeholder groups. 
However, for the purposes of this framework, this measure has been included within the business 
sector tier relating to environmental sustainability and work practices. 

TVET economic and social objectives around ROI 

To develop a foundation for the ROI framework, there is a main ROI objective for each stakeholder 
group. Table 9 summarises how TVET contributes to the economic and social outcomes of stakeholder 
groups which form the focus of the ROI framework. 


Table 9 TVET contribution to economic and social outcomes by stakeholders 


Stakeholder 

Economic 

Social 

Individual 

TVET provides the skills required to 
participate in the labour market. 

TVET contributes to improved social outcomes for 
individuals. 

Business 

TVET meets the needs of business/industry 
outcomes. 

TVET contributes to a healthy, safe and 
sustainable workplace environment. 

Economy/society 

TVET contributes to improved economic 
outcomes in the economy. 

TVET contributes to improved social outcomes in 
society. 


In terms of economic measures, TVET contributes to labour market skills, business outcomes and 
economic growth. TVET social outcomes relate to health and well-being for individuals, employees 
and society, as well as a safe, environmentally sustainable workplace. 

Key ROI indicators 

A review of the research literature on ROI (Griffin 2016) highlights indicators that relate to the three 
stakeholder groups — individuals, business and the economy — and measure the level of impact on 
economic and social outcomes. An overview of the high level ROI indicators are presented separately 
by stakeholder type, although as mentioned earlier in this section they can influence and interact 
across domains. The indicators that follow have been selected on the basis of their prominence in the 
literature, usefulness, practicality and capacity to add value to stakeholder groups along with the 
ability to apply to different types of training and contexts. 

Individuals 

The key economic indicators for individuals in TVET are categorised as job-related measures. This 
primarily focuses on the labour market and employment of those not in a job before training (Karmel 
& Nguyen 2006; Noonan et al. 2010) and productivity in the form of financial returns or higher wages 
(Leigh 2008; Lee & Coelli 2010). An improved employment status as a result of a higher skill level of 
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employment or promotion (Karmel & Fieger 2012) is also included as a valued indicator. The 
attainment of employability skills for those completing training is an effective measure to determine 
the level of non-technical skills and knowledge required to get a job and participate effectively in the 
workforce. Importantly, an entrepreneurship indicator is essential to capture the skills and knowledge 
gained from participation in entrepreneurial education. 

The social return on investment for individuals focuses on non-job-related indicators. An improved 
sense of health and well-being in the form of self-esteem, confidence and life or work satisfaction 
feature prominently in a number of studies (Stanwick et al. 2006; Deloitte Access Economics 2011; 
Cedefop 2013). The attainment of foundation skills following training measures the level of language, 
literacy and numeracy skills and improves individual capacity (Brown et al. 2015). Improved 
socio-economic status reflects positive returns to employment status changes, household income or 
living standards and increased returns to social inclusion measures through participation in social 
groups or communities (Cedefop 2011c; Deloitte Access Economics 2011; Priest 2009). 

Business 

The business stakeholder group includes employers, employees, individual businesses and the industry 
sector. Although employers implement highly specific training aimed at targeted groups to meet 
specific business needs, there are several key economic measures in the form of market indicators 
that have been well researched. These cover productivity (Bernier & Cousineau 2010; Columbo & 
Stanca 2008; Zwick 2006, cited in AWPA 2013), profitability (Blandy et al. 2000; AWPA 2013), quality 
improvement of products and services and business innovation through introducing new technologies 
and work practices (Maglen et al. 2001; Helper et al. 2016). 

Businesses operate similar to small communities and as such, generate social and environmental 
benefits. In particular, employee well-being factors of improved motivation, confidence or job 
satisfaction (Cedefop 2011b) can strengthen workforce and employee engagement (which reduces 
absenteeism and staff turnover) (Cedefop 2013; Kennett 2013) and increase social returns. A safer 
workplace and environmentally sustainable work practices, such as increased recycling and reduced 
waste, are key non-market indicators of business returns. It is important to note that non-market 
returns can impact market indicators and deliver economic payoffs. For example a high level of 
employee engagement may translate into lower absenteeism and increase business productivity. 

Economy and society 

Analysis of ROI at a macro level often employs economic modelling as a predictive tool (Australian 
Productivity Commission 2012; Independent Economics 2013). This often focuses on economic growth as a 
key indicator of TVET’s impact on the economy as measured by gross domestic product (GDP). The level of 
labour market participation, reduced unemployment rates and a skilled workforce indicate other key 
measures. Entrepreneurial activity is yet another measure as it brings value, innovation and employment 
growth to the economy. 

TVET ROI from education and training brings other benefits to society. Research indicates that 
indicators of improved health (Cedefop 2013), increased social cohesion (NVEAC 2011), improved 
social equity (Buddelmeyer et al. 2012) through increased access and participation of disadvantaged 
groups in TVET and strengthened social capital (Cedefop 2011a) through participation in networks 
provide reliable measures of social returns. 
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ROI and public policy 


An additional dimension considered in the literature (but less frequently considered in ROI 
calculations) is the extent to which ROI plays a role in the evaluation of public policies related to 
TVET. However, as pointed out by the European Training Federation (ETF 2008) increasing budgetary 
constraints in the delivery of public services (including education and training) means that 
expenditure must be more strongly defended and justified and the beneficiaries need to be more 
accurately targeted. Returns to public policy or program interventions are often reported in terms of 
cost-effectiveness or relate to the impact of a program to its overall costs. However, there are 
frequently multiple forms of return which interest policy makers including social effectiveness, 
strategic effectiveness and credibility which lend it to impact assessment using a mix of qualitative 
and quantitative approaches (Stufflebeam et al. 2000). 

A practical example is the United Nations Development Program (UNDP) which has established 
indicators around four key policy objectives of TVET: 

■ Participation - considered here as social partners and stakeholders participating in decision making. 

■ Accountability - transparency and governance. 

■ Decentralisation - autonomy in decision making and innovation of the training system. 

■ Effectiveness and efficiency — system outcomes as they apply to labour market needs (Homs 2007). 

ROI framework 

Figure 4 presents a diagram of the return on investment (ROI) framework for each stakeholder group 
and the economic and social indicators. The TVET ROI objectives also feature on the table. Following 
the ROI framework is a detailed description of the key ROI indicators and measures which are shown 
in the following tables: 

■ ROI indicators and measures for Individuals 

■ ROI Indicators and measures for Business 

■ ROI Indicators and measures for Economy/Society. 

The figure also provides a list of example measures developed from analysis of existing studies. 
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Figure 4: Return on investment (ROI) framework 
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ROI indicators and measures for individuals 


Indicator 

Measure 

Example measures 

JOB-RELATED 

1. Employability skills 

Attainment of employability skills for those 
who have completed training. 

Attainment of the non-technical skills and 
knowledge required to get a job and 
participate effectively in the workforce. These 
include communication, 
self-management, planning, teamwork, 
decision making and problem solving skills. 

2. Employment 

Employment rate of those not employed 
before training. 

Proportion of TVET graduates who are 
employed at the end of their training. 


3. Improved employment 
status 


Improved employment status of those 
employed before training 
who have completed training. 


Proportion of TVET graduates who report 
improved employment circumstances. For 
example casual to permanent status, 
part-time to full-time status or promoted to a 
higher level of employment. 


4. Wages/Earnings 


Income of full-time workers after training. 


Earnings of those employed full-time after 
training. As measured by gross net earnings, 
gross monthly earnings, weekly earnings, pre¬ 
tax hourly wages or annual earnings. 


5. Entrepreneurship 


Attainment of entrepreneurial skills and 
knowledge. 


Attainment of skills, knowledge and attitudes 
that aim to build an entrepreneurial mindset 
and skillset required to transform ideas into 
action. 


NON-JOB-RELATED 

1. Well-being 


Students have an improved sense Improved self-esteem, 

of well-being after training. Improved confidence. 

Life/work satisfaction. 

Self-rated health. 

Satisfaction with financial situation. 


2. Foundation skill 
gains 


TVET graduates/completers have Attainment of language, literacy and numeracy 

improved foundation skills following training skills and financial literacy. 

completion. 


3. Socio-economic status Improved socio-economic status of those 

completing TVET programs. 


Proportion of TVET graduates who report 
positive changes to employment status, 
household income, remoteness or living 
standards. 


4. Social inclusion Participation in social groups or 

communities. 


Membership of a club or organisation, social 
network group, volunteering, sporting 
associations or other communities. Civic 
participation. Social interactions. 
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ROI indicators and measures for business 

Indicator Measure Example measures 


MARKET 

1. Increased productivity Increase in the productivity of the 

organisation. 


Value added per hour of labour, value of 
sales per labour hour, items sold per hour of 
labour. 

Management processes/work practices. 


2. Profitability 


Increase in the profitability of the Reduction in costs, 

organisation. Increased sales. 

Reduced supervision time. 
Reduced scrap/wastage. 
Reduced induction costs. 


3. Quality product/service Improvement in the quality of products or 

services. 


Customer service satisfaction. 
Reduction in error/defects rate. 


4. Business innovation Contributing to organisational innovation 

and business practices. 


Introduction of new technologies. 

Best/new business practices. 

Increased efficiency and use of resources. 
Leadership/management practices/culture. 


NON-MARKET 

1. Employee well-being Employees have an improved sense of well- Improved self-confidence. 

being. Improved motivation. 

Improved morale. 

Job/working conditions satisfaction. 
Reduced employee stress. 


2. Employee engagement Employees are more engaged in the 

workplace. 


Skill gains - workplace language and literacy 
skills. 

Engaged in further study. 

Reduced absenteeism. 

Reduced staff turnover. 

Increased retention rates. 


3. Workplace safety 


4. Environmental 
sustainability 


Employees experience a safer workplace 
environment. 


Workplace practices contribute to 
environmental sustainability. 


Reduced injuries. 

Decreased accidents. 

Improved safety records. 

Meeting compliance regulations. 
Meeting licensing requirements. 

Improved energy and fuel management. 
Efficient waste management. 

Increased recycling of materials. 
Reduced energy and water usage. 
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ROI indicators and measures for the economy 


Indicator 


Measure 


Example measures 


ECONOMY 

1. Economic growth 

2. Labour market 
participation 


Increase in the capacity of the economy to 
produce goods and services. 

Increase in the labour market participation 
ofTVET graduates/completers. 


Gross domestic product (GDP) 

Real GDP (labour productivity) 

Labour market participation of TVET 
graduates. 


3. Unemployment rates 


Decrease in the rate of unemployment for Unemployment rate of TVET graduates. 
TVET graduates/completers. 


4. Skilled workforce 


Level of educational attainment of TVET Qualification levels of TVET graduates who 
graduates/completers. complete training by industry group. 

Productivity level. Higher earnings. Skill types 
by industry groups. 


5. Entrepreneurial activity Level of entrepreneurial activity of TVET 

graduates/completers. 


Business start-up rates. 

Job creation/employment growth 
Social enterprise/community development 
Technological innovation/commercialisation 
Enterprise size/growth rate 


SOCIETY 
1. Health 


2. Social cohesion 


3. Social equity 


4. Social capital 


Improve community health and foster a 
longer and better life. 


Improve the well-being, social inclusion and 

values that support 

co-operation within or among groups. 


Increase access and participation 
of disadvantaged groups in TVET. 


Participation in networks that strengthen 
social capital. 


Self-rated health. 

Reduction of chronic health conditions, body 
mass index. 

Mortality/death rates. 

Reducing disparity and avoiding 
marginalisation. 

Crime reduction. 

Civic unrest status. 

Freedom status (political rights, civic 
liberties). 

Participation rate of disadvantaged groups in 
TVET including those with low socio 
economic status, disability, location 
remoteness, cultural/language barriers. 

Number and types of social networks. Active 
citizenship, civic engagement, volunteering or 
member of social network. For example 
participation/member of clubs, neighbourhood 
groups, organisations, political parties. 
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Guidelines to ROI data collection 

What are the considerations of implementing the ROI framework to collect data? To implement the 
ROI framework and collect data requires defining underlying principles, identifying data sources and 
establishing data quality and availability. Listed are a few guidelines to support an initial ROI data 
collection process. 

Principles 

As a first step to collect data based on the ROI framework and indicators, there are a number of 
underlying principles to guide this process. 

1. The key indicators are designed so they are sufficiently specific but general enough to apply 
across training contexts and environments. 

2. The focus is on identifying existing data sources that relate to the ROI indicators. 

3. The approach is to adopt transparency and clearly identify factors that may impact on the type 
and quality of data and acknowledge the potential impact. 

4. Definition of the contextual underpinnings of the data and stakeholder groups is fundamental to 
understanding the ROI landscape. 

Data sources and profiles 

The steps to establish relevant data sources: 

1. Identify the existing data sources that relate to the key ROI indicators and stakeholder groups 
including financial ROI data. 

2. Define the type of data source — for example, administrative dataset, international collection, 
case study, longitudinal survey. 

3. Document a description of the data including the year and source. 

4. Define the scope of the data or profile of the population. 

5. Identify the key data elements — for example, stakeholder demographics, training program, field 
of education. 

6. Record factors that impact on the result and acknowledge the potential impact (refer table 8). 

Data quality and access 

To establish the data quality and an accurate interpretation of the data a number of steps need to be 
considered. 

1. Define the data quality and completeness of the dataset. 

2. Establish the availability of the data and the level of accessibility. 

3. Identify any data limitations. 

4. Record the information gaps or data gaps by stakeholder group and ROI indicators. 

5. Explain any anomalies in the data. 

In summary, to establish an accurate picture of the existing data source requires defining the 
contextual underpinnings of the data source, establishing the data quality and completeness, 
identifying the data gaps and highlighting any data limitations. 
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Conclusions 


Information on return on investment to training is generally viewed as a valuable tool in arguing the 
case for funding (or additional funding) of TVET systems and programs. Examination of the research 
provides some evidence of positive outcomes from investment in TVET which range from labour 
market or employability benefits through to social and environmental sustainability perspectives. 

The research indicates the importance of ensuring that any measurement of ROI should be closely 
aligned to the objectives of the TVET system of a country. ROI measures related to the economic 
outcomes of TVET (for example, employment related outcomes) are seen as being critically important 
across all countries. Social aspects are also important, and tie into the objectives of the systems, but 
these social measures are often indirectly linked to the economic ones requiring more evidence to 
establish the link. For example, a reduction in crime among young people was seen to be linked to 
improvements in young people in employment. 

Finally, measuring ROI in a given country has its challenges but the diversity of TVET systems and the 
differing contexts of the countries pose considerable difficulties for cross-country comparisons of ROI. 
A challenge is to develop measures of ROI that can be compared across countries, while another is 
having appropriate data to enable the measurement of ROI. This challenge has several aspects to it 
including the need to have ROI in mind when setting up data collection and reporting systems and 
linking it back to the objectives of the TVET system. 
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